Gesture Modelling for Linguistic Purposes
نویسنده
چکیده
The study of sign languages attempts to create a coherent model that binds the expressive nature of signs conveyed in gestures to a linguistic framework. Gesture modelling offers an alternative that provides device independence, scalability and flexibility for the annotation and modelling of linguistic phenomena. This paper presents the requirements and initial experiments to build an input method editor for sign languages. The objective is to design interfaces backed up by computational methods that can infer and use linguistic guidance to model sign language gestures. This in turn produces a linguistically annotated corpus of gesture animations.
منابع مشابه
Gesture Theory is Linguistics: On Modelling Multimodality as Prosody
Prosody and gesture have, with few exceptions, not appealed to computational linguists, and there is even less awareness of the parallels between them, though in modelling sign languages of the hearing-impaired, the linguistic metaphor of gesture patterns as ‘phonology’ has been very fruitful. On the other hand, many disciplines from anthropological linguistics to robotics are currently occupie...
متن کاملToward a Motor Theory of Sign Language Perception
Researches on signed languages still strongly dissociate linguistic issues related on phonological and phonetic aspects, and gesture studies for recognition and synthesis purposes. This paper focuses on the imbrication of motion and meaning for the analysis, synthesis and evaluation of sign language gestures. We discuss the relevance and interest of a motor theory of perception in sign language...
متن کاملCombined Gesture-Speech Analysis and Synthesis
Multi-modal speech and speaker modelling and recognition are widely accepted as vital aspects of state of the art human-machine interaction systems. While correlations between speech and lip motion as well as speech and facial expressions are widely studied, relatively little work has been done to investigate the correlations between speech and gesture. Detection and modelling of head, hand and...
متن کاملRecognition of Arm Movements
This paper describes a method for detection, tracking and recognition of lower arm and hand movements from color video sequences using a linguistic approach driven by motion analysis and clustering techniques. The novelty of our method comes from (i) automatic arm detection, without any manual initialization, foreground or background modeling, (ii) gesture representation at different levels of ...
متن کاملHow to Bootstrap a Human Communication System
How might a human communication system be bootstrapped in the absence of conventional language? We argue that motivated signs play an important role (i.e., signs that are linked to meaning by structural resemblance or by natural association). An experimental study is then reported in which participants try to communicate a range of pre-specified items to a partner using repeated non-linguistic ...
متن کامل